ExM: High-level dataflow programming for extreme-scale systems

نویسندگان

  • Timothy G. Armstrong
  • Justin M. Wozniak
  • Michael Wilde
  • Ketan Maheshwari
  • Daniel S. Katz
  • Matei Ripeanu
  • Ewing L. Lusk
  • Ian T. Foster
چکیده

We present here the ExM (extreme-scale many-task) programming and execution model as a practical solution to the challenges of programing the higher-level logic of complex parallel applications on current petascale and future exascale computing systems. ExM provides an expressive, high-level functional programming model that yields massive concurrency through implicit, automated parallelism. It comprises a judicious integration of dataflow constructs, highly parallel function evaluation, and extremely scalable task generation. It directly addresses the intertwined programmability and scalability requirements of systems with massive concurrency, while providing a programming model that may be attractive and feasible for systems of much lower scale. We describe here the benefits of the ExM programming and execution model, its potential applications, and the performance of its current implementation.

برای دانلود رایگان متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Productive composition of extreme-scale applications using implicitly parallel dataflow

In every decade since the 1970’s, computer scientists have re-examined dataflow-based execution models, hoping the programming productivity benefits these models promise can be realized on practical hardware platforms to implement useful applications. Based on the recent Swift/T implementation of “implicitly parallel functional dataflow” (IPFD) for extremescale systems, we believe that the data...

متن کامل

Topological Patterns for Scalable Representation and Analysis of Dataflow Graphs

Tools for designing signal processing systems with their semantic foundation in dataflow modeling often use high-level graphical user interfaces (GUIs) or text based languages that allow specifying applications as directed graphs. Such graphical representations serve as an initial reference point for further analysis and optimizations that lead to platform-specific implementations. For large-sc...

متن کامل

EFFICIENT DESIGN OF EMBEDDED SIGNAL PROCESSING SYSTEMS USING TOPOLOGICAL PATTERNS BASED DATAFLOW GRAPH REPRESENTATIONS by

Tools for designing signal processing systems with their semantic foundation in dataflow modeling often use high-level graphical user interfaces (GUIs) or text based languages that allow specifying applications as directed graphs. Such graphical representations serve as an initial reference point for further analysis and optimizations that lead to platform-specific implementations. For large-sc...

متن کامل

Automatic Optimization of Parallel Dataflow Programs

Large-scale parallel dataflow systems, e.g., Dryad and Map-Reduce, have attracted significant attention recently. High-level dataflow languages such as Pig Latin and Sawzall are being layered on top of these systems, to enable faster program development and more maintainable code. These languages engender greater transparency in program structure, and open up opportunities for automatic optimiz...

متن کامل

Building a HighLevel Dataflow System on top of MapReduce: The Pig Experience

Increasingly, organizations capture, transform and analyze enormous data sets. Prominent examples include internet companies and e-science. The Map-Reduce scalable dataflow paradigm has become popular for these applications. Its simple, explicit dataflow programming model is favored by some over the traditional high-level declarative approach: SQL. On the other hand, the extreme simplicity of M...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

عنوان ژورنال:

دوره   شماره 

صفحات  -

تاریخ انتشار 2012